Rapid developments in computing and communication technologies are bringing fundamental changes to multimedia applications as well as multimedia systems. On the one hand, remote servers and the “cloud” integrate large amounts of computing and storage resources. Note that the term “cloud” is generally understood to refer to cloud computing environments, such as the Internet, that provide a supplement, consumption, and delivery model for IT services based on Internet protocols, and it typically involves provisioning of dynamically scalable and often virtualized resources. By leveraging such resources in view of increasingly higher available user bandwidth, multimedia systems are able to deliver rich multimedia content to client devices, interactively or even in real-time. On the other hand, client devices continue to become more and more diverse in terms of functionalities and capabilities. The number of client devices each user owns is also tending to increase over time. Consequently, when designing multimedia systems, both the adaptation of heterogeneous computing devices and potential collaboration between such devices should be considered to provide a better user experience.
Conventional virtual display techniques are a good example of device collaboration, in which the computing is distributed among local clients and remote instances. Typically, most computations for these virtual display techniques are carried out remotely utilizing remote computing resources, and only high-level or low-level drawing primitives are transmitted to the local clients. Local clients then use these drawing primitives to render the final display on the client device. In general, these types of virtual display architectures are very suitable for thin-client computing systems where the local client and the remote instance communicate via a point-to-point connection. Consequently, such architectures have been widely adopted to support various multimedia applications.
Unfortunately, traditional virtual display architectures are not well suited to meet many of the new requirements raised by the increasing popularity and availability of heterogeneous computing devices. For example, heterogeneous computing devices, such as workstations, desktop PCs, laptops, and a wide variety of handheld or mobile devices, make different trade-offs between mobility, display capability, and user input mechanism. Therefore, such devices provide different user experiences based on those trade-offs. In general, small-size devices that can be taken anywhere, such as mobile phones, are widely adopted by users. However, given their generally relatively small size, these portable devices often suffer from compromised display area and inefficient user input capabilities. In contrast, dedicated IO resources, such as large displays, touch screens, projection devices, etc., are usually dedicated to particular devices with poor mobility (e.g., such devices are too large, too heavy, and require too much power to carry in the user's pocket or purse.